In processing a digital image, it is common to sharpen the image and enhance fine detail with sharpening algorithms. Typically, sharpening is performed by a convolution process (for example, see A. K. Jain, Fundamentals of Digital Image Processing, Prentice-Hall: 1989, pp. 249–251). The process of unsharp masking is an example of a convolution-based sharpening process. For example, sharpening an image with unsharp masking can be described by the equation:s(x,y)=i(x,y)**b(x,y)+βf(i(x,y)−i(x,y)**b(x,y))   (0)where:    s(x,y)=output image with enhanced sharpness    i(x,y)=original input image    b(x,y)=lowpass filter    β=unsharp mask gain factor    f( )=fringe function    ** denotes two dimensional convolution    (x,y) denotes the xth row and the yth column of an image
Typically, an unsharp image is generated by convolution of the image with a lowpass filter (i.e., the unsharp image is given by i(x,y)**b(x,y)). Next, the highpass, or fringe data is generated by subtracting the unsharp image from the original image (i.e., the highpass data is found with i(x,y)−i(x,y)**b(x,y)). This highpass data is then modified by either a gain factor β or a fringe function f( ) or both. Finally, the modified highpass data is summed with either the original image or the unsharp image to produce a sharpened image.
A similar sharpening effect can be achieved by modification of the image in the frequency domain (for example, the FFT domain) as is well known in the art of digital signal processing.
One problem associated with image sharpening is noise amplification. Noise amplification can be a special problem when sharpening a digital image originating from film. Specifically, image regions originally receiving little or no light exposure (also called Dmin) on the film can appear quite noisy when the image is sharpened.
Curlander (in “Image Enhancement Using Digital Adaptive Filtering,” Master's Thesis, Massachusetts Institute of Technology, 1977, p. 30–34, 48–49, 63,72–73, 93–94) describes methods of sharpening a digital image, where the highpass gain factor is determined from the lowpass signal value, the local contrast, or the highpass signal value. Curlander did not describe sharpening of a rendered image in such a way as to minimize the amplification of image Dmin noise.
For photographic negatives, the areas of the film receiving no exposure have a minimum density called Dmin. Dmin is also sometimes referred to as mask or base density. Underexposed film images typically have image areas containing Dmin. It is common to use the value of Dmin in the processing of digital images for the purpose of improving image quality. For example, in U.S. Pat. No. 5,081,485 issued Jan. 14, 1992, Terashita describes a method of using a mask density to improve the exposure estimate for an image. To this end, the mask density is subtracted from the average density. This increases the robustness of the determined color balance by decreasing the variability between different color negative films. However, Terashita's method does not ensure that regions of an image sensing device receiving little or no light exposure are specially handled to reduce the sharpening of noise.
It is occasionally desirable to sharpen different regions or pixels of the image by different amounts. For example, is it has been suggested that it is desirable to sharpen the pixels representing human faces to a lesser degree than pixels representing a building. For example, in U.S. Pat. No. 5,682,443 issued Oct. 28, 1997, Gouch et al. describe the modification of the gain of the unsharp mask based on the color of a pixel (and the color of the surrounding neighborhood). Gouch does not consider the undesirable noise amplification of Dmin regions that accompanies the image sharpening.
Alternatively, in U.S. Pat. No. 4,571,635 issued Feb. 18, 1986, Mahmoodi et al. teach a method of deriving a gain factor β that is used to scale the high frequency information of the digital image depending on the standard deviation of the image pixels within a neighborhood. In addition, in U.S. Pat. No. 5,081,692 issued Jan. 14, 1992, Kwon et al. teach that a gain factor β is based on a center weighted variance calculation. In U.S. Pat. No. 4,761,819 issued Aug. 2, 1988, Denison et al. describe a method where the gain factor of an unsharp mask is dependent on both a local variance calculation and a noise statistic. While these methods do indeed sharpen the image while attempting to minimize noise amplification, they are computationally complex. In addition, these methods do not explicitly consider the Dmin region of the image, and therefore some Dmin noise is amplified.
Shimazaki in U.S. Pat. No. 5,051,842 issued Sep. 24, 1991, describes an apparatus which generates unsharp signals from images, derives two parameters based on either the image signal level or the unsharp signal level from a pre-determined lookup table, multiplies one parameter with the image signal, multiplies the other parameter with the unsharp signal, and adds the two resulting signals to obtain the final image signal. One embodiment requires that the sum of the two parameters equal one for all image signal levels. In this case, the method is mathematically equivalent to the unsharp mask equation. Shimazaki teaches that the two parameters are signal dependent with the signals representing image highlights resulting in the highest degree of sharpening. The two parameters are chosen such that the sharpening decreases as either the image signal or the unsharp signal decreases until the sharpening level is zero. At that point, the sharpening converts to blurring as the image signal or unsharp signal continue to decrease into the shadow region of the density range. Shimazaki's apparatus suffers from not accounting explicitly for the Dmin density level.
Gallagher et al. in U.S. Pat. No. 6,167,165 issued Dec. 26, 2000, describe a method of selecting a gain for an unsharp mask based on a local intensity level. While this method does demonstrate an unsharp mask gain having dependence on local intensity, is does not describe sharpening in such a manner as to de-emphasize Dmin noise.
Keyes et al. in U.S. Pat. No. 6,091,861 issued Jul. 18, 2000, and Matama in U.S. Pat. No. 6,384,937 issued May 7, 2002, both describe methods of selecting a constant, position independent gain factor based on the exposure of the image. Lower gain factors will be selected for images that are underexposed, thereby providing less gain for that image than for a normally exposed image. These methods are insufficient to handle scenarios where an image contains both underexposed (or Dmin) regions and normally exposed regions. Since a single gains factor is selected for the entire image, either the underexposed region or the normally exposed region will be sharpened with a sub-optimal gain factor.
Further complicating the problem of Dmin noise being amplified by sharpening is that it is typical for an imaging system to sharpen a rendered image. Rendering, or mapping input image densities to output media densities on the output media occurs in both digital imaging and optical imaging and is well known to those skilled in the art. U.S. Pat. No. 6,097,470 issued Aug. 1, 2000 to Buhr et al., describes image rendering. It is difficult to determine the areas of a rendered digital image that correspond to the Dmin of the film. This is because a black portion of a rendered digital image could originate from a Dmin portion of the film, or it could originate from a normally exposed portion of the film. Thus, it is difficult to avoid amplifying the Dmin noise when sharpening a rendered image. U.S. patent application Ser. No. 09/981,176, filed Oct. 17, 2001, describes a method of determining the propagated value of Dmin in a rendered image, but there is no mention of using that propagated Dmin value to aid in the sharpening of the rendered image. In addition, none of the previously mentioned sharpening methods describes sharpening a rendered image such that the Dmin noise is not amplified.
Therefore, there exists a need for an improved image sharpening method that adjusts the amount of sharpening while avoiding amplifying Dmin noise.